Strongly Typed Inductive Concept Learning

In this paper we argue that the use of a language with a type system, together with higher-order facilities and functions, provides a suitable basis for knowledge representation in inductive concept learning and, in particular, illuminates the relationship between attribute-value learning and inductive logic programming (ILP). Individuals are represented by closed terms: tuples of constants in the case of attribute-value learning; arbitrarily complex terms in the case of ILP. To illustrate the point, we take some learning tasks from the machine learning and ILP literature and represent them in Escher, a typed, higher-order, functional logic programming language being developed at the University of Bristol. We argue that the use of a type system provides better ways to discard meaningless hypotheses on syntactic grounds and encompasses many ad hoc approaches to declarative bias. 1. Motivation and scope Inductive concept learning consists of finding mappings of individuals (or objects) into discrete classes. Individuals and induced mappings are represented in some formal language. Historically, attribute-value languages (AVL) have been most popular in research in machine learning. In an attribute-value language, individuals are described by tuples of attribute-value pairs, where each attribute represents some characteristic of the individuals (e.g., shape, colour, etc.). Although very useful, attribute-value languages are also quite restrictive. In particular, it is not possible to induce relations explicitly in that framework. In recent years, researchers have thus proposed the use of first-order logic as a more expressive representation language. In particular, the programming language Prolog has become the almost exclusive representation mechanism in inductive logic programming (ILP). The move to Prolog alleviates many of the limitations of attribute-value languages. However, traditional application of Prolog within ILP has also caused the loss of one critical element inherent in attribute-value languages: the notion of type. Implicitly, each attribute in AVL is a type, which can take on a number of possible values. This characteristic of AVL makes it possible to construct efficient learners since the only way to define mappings of individuals to classes consists of constructing expressions (e.g., conjunctions) that extract a particular attribute (i.e., tuple projection) and tests its value against some value of that type. On the other hand, Prolog has no type system. All characteristics of individuals are captured by predicates. As a result, the way to construct mappings becomes rather unconstrained and a number of ad hoc mechanisms (e.g., linked clauses, mode declarations, determinacy, etc.) have to be introduced to restore the tractability of the learning problem. A major aim of this paper is to demonstrate the usefulness of a strongly typed language for inductive concept learning, where we define a strongly typed language as one having a fully-fledged type system. We also propose that the natural extension of attribute-value learning to firstand higherorder consists in representing individuals by terms. Hence, individuals may have arbitrary structure, including simple tuples (as in the attribute-value learning case), lists, sets and indeed any composite type. This provides us with a unified view on attribute-value learning and inductive logic programming, in which a typed language such as Escher1 (a typed, higher-order, functional logic programming language being developed at the University of Bristol) acts as the unifying language (Fig. 1). At Bristol, we have begun the implementation of a decision-tree learner, generalised to handle the constructs of the Escher language. Several of the illustrative examples below were run on this learner and the results produced are reported here. Full details about the learning system, its implementation, and the results of some largerscale practical experiments will be reported elsewhere. The paper is organised as follows. Section 2 describes Escher, the programming language which serves as a vehicle for the implementation of the aforementioned extension. Section 3 contains a number of illustrative learning tasks reformulated in Escher. In Section 4 we discuss the main implications of our approach for inductive logic programming, and Section 5 contains some conclusions. 1 M.C. Escher is a registered trademark of Cordon Art B.V., Baarn, Nederland. Used by permission. All rights reserved. complexity of representation of individuals AVL Esche r Prolog A-V le arning ILP tuples of constants tuples, lists, sets, constants Fig. 1. The relation between attribute-value learning and ILP is illuminated by viewing it through a strongly typed language such as Escher. One of the main differences lies in the complexity of the terms representing individuals. 2. Elements of Escher This section highlights the main features of the Escher language [3] and assumes familiarity with Prolog. We mainly deal with list-processing functions. It should be noted that the syntax of Escher is compatible with the syntax of Haskell (a popular and influential functional programming language). Consequently, Escher programs may look uncomfortably unfamiliar to Prolog aficionados (lowercase variables, constants starting with a capital), but we hope the reader will be able to abstract away from syntax. The Escher definition of list membership is as follows (y:z stands for the list with head y and tail z, == is the equality predicate, and || is disjunction). member::(a,[a])->Bool;